Using x-gram for efficient speech recognition

X-grams are a generalization of the n-grams, where the number of previous conditioning words is different for each case and decided from the training data. X-grams reduce perplexity with respect to trigrams and need less number of parameters. In this paper, the representation of the x-grams using finite state automata is considered. This representation leads to a new model, the non-deterministic x-grams, an approximation that is much more efficient, suffering small degradation on the modeling capability. Empirical experiments for a continuous speech recognition task show how, for each ending word, the number of transitions is reduced from 1222 (the size of the lexicon) to around 66. 1. REVIEW OF X-GRAMS One of the main components in continuous speech recognition is the language model. The model has to estimate the a priori probability of each possible sentence. The most extended models are based on n-grams, basically bigrams or trigrams, which estimate the probability of each word taking into account only the previous n-1 words. Recently, the authors have introduced xgrams [1]. The difference between n-grams and x-grams is that, in x-grams, the length is not fixed by the designer of the language model, but found automatically by the algorithm from the training data. For each sequence of given words, the method decides the number of words which really conditionate the probabilities of the incoming words. To decide if a history is relevant or not, the algorithm uses the number of times that the history occurs in the training data and the divergence function [1]. X-grams are estimated using back-off smoothing. X-grams are more efficient than trigrams in the following sense: x-grams are more compacts (smaller) than trigrams and produce smaller perplexity. This is because some probabilities distributions conditioned by the two previous words do not improve the estimation with respect to the use of only the one previous word. On the other hand, in other cases, the estimation of the probabilities conditioned by the 3, 4 or, in general, the x previous words is significantly better that the estimation considering only the two previous words. To illustrate this, table 1 shows the perplexity and the number of histories considered in trigrams and with x-grams. The task consists in queries to a database with geographical information (rivers, mountains, cities, etc.). The models are trained using 8262 queries (90.000 words) and the perplexity is evaluated using 1147 queries. All the queries in the training and in the test are different. The size of the lexicon is 1222 words. This research was supported by CICYT, contract TIC95-0884-C04-02 Model Perplexity # histories